RBUE: a ReLU-based uncertainty estimation method for convolutional neural networks

Abstract Convolutional neural networks (CNNs) have successfully demonstrated their powerful predictive performance in a variety of tasks. However, it remains challenge to estimate the uncertainty these predictions simply and accurately. Deep Ensemble is widely considered state-of-the-art method which can accurately, but expensive train test. MC-Dropout another popular that less costly lacks diversity resulting accurate estimates. To combine benefits both, we introduce ReLU-Based Uncertainty Estimation (RBUE) method. Instead using randomness Dropout module during test phase (MC-Dropout) or initial weights CNNs (Deep Ensemble), RBUE uses activation function obtain diverse outputs testing uncertainty. Under method, propose strategy MC-DropReLU develop MC-RReLU. The uniform distribution function’s position allows be well transferred output results gives more output, thus improving accuracy estimation. Moreover, our simple implement does not need modify existing model. We experimentally validate on three used datasets, CIFAR10, CIFAR100, TinyImageNet. experiments demonstrate has competitive favorable training time.